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SMEAR REDUCTION IN CCD IMAGES 

This invention relates to charge coupled devices 
(CCDs) and in particular the improvement of CCD 
5 performance at low incident light levels, 

EP-A-0,866,501 describes the use of a multiplication 
register which uses impact ionisation to amplify the 
charge stored in an output register. The technique 
described in that patent application is extremely 
10 effective and can produce very high gains with little 
noise. Consequently, devices using multiplication 
registers are well suited for operation at very low 
incident light levels. 

One artefact of the multiplication register is 
15 referred to as horizontal smearing. This occurs when 
charge is not fully transferred from one cell of the 
register to the next. Where an image acquired in very low ^ 
light levels includes a bright object, such as a car & 
headlight at night, the localised bright source will tend 
20 to be smeared across the image as charge which has not 

been transferred properly is amplified on subsequent clock 
pulses. In a generally bright scene, such residual charge 
may not be noticeable, but in a generally low light level 
image it can produce an undesirable streak. 
25 A similar streaking can be produced in the vertical 

direction as a consequence of the manner in which images 
are acquired and then clocked out. In the images described 
in EP-A-0, 866, 501 the image comprises an area array sensor 
which acquires light simultaneously for all pixels of the 
30 array. After a set integration time, the charge stored is 
clocked out line by line to a store. 

Where the device is operating in very low light 
levels, localised bright sources tend to be smeared 
vertically as, during the clocking out process, which 
35 happens from the bottom of the array, lines above the 

localised bright source are exposed to that bright source 
for a short time as they are clocked past the source. 



Although the exposure time is small compared to the 
integration time, the localised bright source is so much 
brighter than the background that the light captured shows 
up on the image as a vertical streak. This streak will 
extend both below and above the localised bright source, 
the streak above having been contributed by the clocking 
out of that frame and the streak below by the clocking out 
of the previous frame. 

The problem occurs whether or not the lines of 
accumulated charge are clocked directly to an output 
register or via an intermediate store. It is a problem 
that is inherent in any CCD that reads out images line by 
line, and is not related to the use of a multiplication 
register. 

One approach to reducing vertical smear, also known 
as frame transfer smear, is to block the light to the CCD 
array during frame transfer. This may be achieved by 
introducing a shutter into the optical path which is 
synchronised to the frame transfer. However, practical 
shutters have limited performance and the most suitable 
shutters are affected by changes in temperature. 

The invention, in its various aspects, aims to 
address the problems of horizontal and vertical smear. 

Broadly, a first aspect of the invention compensates 
for vertical and/or horizontal smears by using edge 
detection techniques to locate smeared areas. Data is then 
substituted into these smeared areas . 

More specifically, there is provided apparatus for 
compensating image signals produced by a CCD imager for 
smears, comprising: an image data analyser for detecting 
the boundaries of a smear; and an image data replacer for 
replacing data between the boundaries of a detected smear 
with alternative image data. 

Preferably, the image data analyser comprises means 
for detecting the boundaries of a smear by detecting rates 
of change in the image data greater than a predefined 
limit 



Embodiments of the invention have the advantage that 
smears can be -simply detected and corrected for as part of 
the signal processing performed on acquired image data. In 
one embodiment of the invention, the alternative data used 
for substitution is derived from image pixels neighbouring 
the detected smear. In another embodiment, the alternative 
data is derived by interpolation which has the advantage 
of looking more natural. 

Preferably, the image data analyser analyses the • 
image line by line to detect intensity gradients greater 
than the predefined limit and so indicative of a smear. 
The analysis may be performed using a kernel to analyse a 
portion of the image data at a time. 

The kernel may be used with a sliding window moveable 
across the image to analyse the complete image. 

Preferably, a temporal integrator integrates two*or 

more images acquired by the CCD imager prior to boundary 

detection by the image data analyser. This has- the .'-A. 

r 

advantage of eliminating random noise, for example caused 
by thermal effects, prior to boundary detection. & 

Preferably, a coordinate extractor extracts the 
coordinates of detected smear boundaries and provides 
these coordinates to the image data replacer. 

Embodiments of this aspect of the invention have the 
advantage that they can be used to detect and compensate 
for vertical frame transfer smear which arises as lines 
are read out from the active image area. It may also be 
used to compensate for horizontal smear caused by 
incomplete charge transfer in a multiplication register. 
Embodiments of the invention are particularly suited to 
use in low light level environments" where the transitions 
between dark area and smeared signals have 4; large 
gradient. 

A second aspect of the invention compensates for 
frame transfer smear by deriving an error signal 
indicative of charge accumulated during read out of a 
line. This error signal is then subtracted from the actual 
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line signals. Masked lines of the image, which are the 
last to be read out, and which are only exposed to 
incident light during frame transfer, may be used to 
derive the error signal. 

5 More specifically, there is provided apparatus for 

compensating for frame transfer smear in a CCD imager, the 
imager comprising an image area having a plurality of rows 
of pixels for accumulating charge during an image 
acquisition period, each row corresponding to a line of 

10 the image, an output register for receiving accumulated 
charge row by row during a transfer mode to produce line 
signals, and at least one row of pixels masked from 
incident radiation and arranged on the side of the image 
area opposite the side on which pixels are transferred to 

15 the output register, the apparatus comprising a subtractor 
for subtracting a signal corresponding to charge 
accumulated during the transfer mode and transferred to 
the output register as lines corresponding to the at least 
one masked row, from the line signals corresponding to the 

20 rows of the image area. 

Embodiments of this aspect of the invention have the 
advantage that frame transfer smears can be compensated 
for reliably where pixels of the image have not been 
saturated and detail in the image is recovered. It may be 

25 used in conjunction with the first aspect of the 
invention. 

Preferably the CCD sensor comprises a plurality of 
masked rows. The error signal may be generated from an 
• average of the masked row line signals acquired during the 
30 frame transfer period. 

A third aspect of the invention increases the dynamic 
t range of the CCD imager by varying the gain on p^rts of an 

image or selected images. For example, the gain may vary 
between two* levels on alternate lines or frames . 
35 More specifically, there is provided apparatus for 

controlling a CCD imager, the imager comprising an image 
area for acquiring charge from incident radiation, an 
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output register for receiving acquired charge from the 
image area during a transfer mode, and a multiplication 
register for multiplying charge received from the output 
register, the apparatus comprising a gain controller for 
5 controlling the gain of the multiplication register to 

vary the gain of the multiplication register for selected 
images or portions of images. 

Preferably, the gain is alternatively low and high on 
alternative lines or images although other low/high ratios 

10 may be used. 

Embodiments of this aspect of the invention have the 
advantage that the dynamic range of the CCD imager may be 
extended thus allowing detail to be presented in both dark 
and light areas of the image. This in turn improves the 

15 usefulness of the smear compensating techniques of the 

first and second aspects of the invention. ° 

•t 

Embodiments of the invention will now be described, 
by way of example only, t ar>d with reference to the £ 
accompanying drawings, in which: 
20 Figure 1 is a schematic view of a known CCD image 

having a multiplication register; 

Figure 2 is a schematic view of a horizontal and 
vertical smear reduction circuit; 

. Figure 3 is a .schematic view of a subtraction circuit 
25 embodying a second aspect of the invention; and 

Figure 4 is a similar view to Figure 1, modified 
according to a third aspect of the invention. 

The CCD device 1 illustrated in figure 1 comprises an 
image area 2, a store section 3 and an output or read-out 
30 register 4. The output register 4 is extended in a linear 
direction to provide a multiplication register 5, whose 
output is connected to a charge detection circui-t 6. At 
the top of the image area 2 is a number, for example 3, of 
masked rows 8 which are shielded from incident radiation. 
35 It is commonplace to have such masked rows in a CCD array 
although they are often located at the bottom of the 
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array. The remaining active image area comprises several 
hundred lines of pixels, for example 288 lines. 

During operation, radiation incident at the active 
lines of the image area is converted to signal charge, the 
5 magnitude of which is representative of the intensity of 
the radiation imposing on the pixels of the active image 
array. After an image acquisition period of typically 
20ms, drive pulses are applied to control electrodes 10 
and 12 to transfer the charge accumulated at the active 
10 pixels to the store section 3. The pixels are transferred 
line by line with a typical transfer time of l^s per line, 
with the bottom line being transferred out first. 

When a complete line has beentransferred, drives 
signals are also applied to control electrodes 12 of the 
15 store to cause charge to be transferred line by line to 

the output register 4. When a row of charge is transferred 
to the output register, drive pulses are applied to 
control electrodes 14 of the output register to transfer 
charge from the elements of the output register to the 
20 multiplication register 5. Charge multiplication is 

achieved in the multiplication register by applying high 
amplitude drive pulses via multiplication register 
electrodes 17 to transfer charge from one element to the 
next and to increase the amount of charge by impact 
25 ionisation. Each signal charge packet stored at the output 
register 4 undergoes an identical multiplication process 
as it proceeds through the elements of the multiplication 
register. 

Referring now to figure 2, a first aspect of the 
30 invention aims to eliminate horizontal and/or vertical 
smearing by using edge and/or gradientdetection to 
identify smeared areas and then to remove the smeared 
areas and replace them with image data substituted from 
elsewhere, or interpolated image data. 
35 The processing of the signal using the circuit of 

figure 2 is performed on the output from the charge 
detection circuit 6. Thus, the input to the smear 
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reduction circuit of figure 2 is a digital video signal 
generated from the CCD imager of figure 1. The input 
signal is first passed to a temporal integrator 20 which 
is connected to a field or frame store 22. The temporal 
5 integrator reduces the effects of random noise which is 
particularly visible at low light levels and averages 
signals -over a number of fields. The term field is used 
here to refer to one image captured by the CCD imager. 
Where the output is a conventional 2:1 interlace video 

10 signal , one image captured by the CCD imager will form a 

field of the video signal. However, if no interlacing were 
used, an image captured by the CCD imager would form a 
frame of the output video signal. 

The output of the temporal integrator is passed to a 

15 spatial kernel processor 24 which itself is coupled to a 
sliding window mechanism 26. The kernel processor and 
sliding window are used to search the image or a specified'*;, 
area of the image to detect vertical or horizontal smears ^ 
and relies on the fact that the smear will produce a large / r 

20 rate of change between its boundaries compared with that r 
of neighbouring real image pixels. In its simplest form, 
the kernel processes the image on a line by line basis 
searching for a gradient that exceeds predefined 
thresholds. This detects one edge of the smear on that 

25 line. The other edge is found by continuing .until a 

negative gradient is found that exceeds the predefined 
threshold. The sliding window mechanism 26 is used to 
provide an efficient identification of the smeared areas 
and defines a vertical edge detector mask. This may be a n 

30 x n array of multipliers, n may typically be 3, which is 
moved across the image to multiply the image in groups of 
pixels . Other methods of edge or transition detection may 
be used. 

A coordinate extractor 28 extracts and stores the 
35 coordinates of the smeared edges. The information masked 
by the optical overload must then be regenerated. If the 
CCD pixels have been saturated then no information can be 
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retrieved from the smear area and so an approximation to 
the original image must be made. This task is performed by 
a data replacement unit 30. The data replacement unit may 
use any conventional data replacement technique. For 
example it may average neighbouring pixels to produce an 
approximation of the smeared area or it may interpolate 
the intensity data between the edges of the smear. The 
latter is presently preferred as it results Ina more 
natural image as a whole.' 

The circuit of figure 2 may also be used for a 
horizontal smear detection and compensation by 
concentrating on the horizontal axis to detect the height 
of the horizontal smear and correct it on the manner 
described. Typically, the two types of smear detection and 
compensation will be performed one after the other. 

Referring back to figure 1, the CCD array includes a 
number of masked rows of CCD elements which do not receive 
any incident light. In. a second aspect of the invention, 
these rows are used to compensate for vertical, or frame 
transfer, smear either instead of or in addition to the 
compensation technique referred to with reference to 
figure 2. 

The masked lines are typically used to correct for 
thermal noise. However, as they are located at the top of 
the array, they can be used to derive a signal that also 
compensates for vertical smear. The charge accumulated in 
each row of the array is passed row by row to the output 
register 4. The contents of the dark or masked rows may 
also be passed to the output register. As these rows are 
at the top of the array, they will be clocked line by line 
through the active array during which time they will 
accumulate charge. This accumulated charge is u 
approximately equal to the vertical smear and can be used 
to derive a correction signal which can be subtracted from 
the signal generated from the active rows of the area 
array. .JThis is possible as the smear is constant both in 
position and intensity from row to row. 
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An integration time of 20 ms was previously mentioned 
for image acquisition. This is typical for CCIR based TV 
systems. The transfer rate is typically 1/is per line. 
Thus, if a bright object illuminates 10 pixels vertically, 
5 the corresponding pixels in the rows above the bright ■ ■ 
object, including the masked rows will accumulate charge 
from that bright object for 10^s. Thus, if the bright 
source is less than 2000 times brighter than the average 
scene it will still be possible to recover information by 

10 subtracting the smear signal from all pixels. This is 

• based on the requirement that the smeared signals are not 
saturated so removing the ability to compensate by 
subtraction of the derived smear signal. In practice, this 
technique works at even higher levels of overload as the 

15 average illumination level is usually less than CCD 

saturation. Similarly, if the bright source illuminates %> 
more than 10 pixels, the level of overload before 
saturation will be reduced as the time exposed to the f 
bright source increases. 

K 

20 Where saturation occurs, the technique described wi?th 

respect to figure 2 may be used to compensate further for* 
vertical smear. - - 

Thus, in the embodiment of this aspect of the 
invention illustrated in figure 3, the output from the 

25 charge multiplier 5 (fig 1) is passed via amplifier 6 and 
a digitiser 39 to a switch which, for example under the 
control of a counter, passes the masked lines to a first 
store 42 and the active lines to an active line frame 
store 44. It is necessary to store the active lines as the 

30 masked lines are read out after the active lines: The 

output from the masked line store is then subtracted from 
the output of the active line store by subtractor 4 6 which 
is an n-bit parallel subtractor, where n is the number of 
bits per pixel. The error signal may be simply the signal 

35 stored for one dark row or may be the average of the. pixel 
values for two or three rows. The output of the frame 
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store 44 is read out sequentially line by line so that the 
error signal is subtracted from each line in turn. 

In both the embodiments of the first anc* second 
aspects of the invention described above, smear 
5 compensation will only be effective where there are 

relatively limited levels of overload. It is therefore 
desirable to extend the dynamic intra-scene range of the 
CCD imager. A third aspect of the invention varies the CCD 
multiplication gain on a line by line or field (image) by 
10 field basis. This enables the intra-scene dynamic range to 
be extended greatly. Using the image of figure 1, the 
range may be varied by 500-1000 times. 

Referring to figure 4, a modification of figure 1 is 
shown in which the gain control is, itself, controlled by 
15 a switch 50 which is switched every line, for example by a 
system clock which controls the voltages to the 
electrodel7. This may be switched every line so that odd 
lines are set to a gain of, for example, 1000 whereas for 
even numbered lines the gain is set to 1. On odd lines, 
20 details in the darkest parts of the image are displayed 
but bright sources would be clipped due to the limited 
capacity of the multiplication register to amplify the 
charge generated by bright sources. At the even lines, 
there will be insufficient multiplication to display dark 
25 areas of the image but detail in bright areas can be seen. 
The alternate lines can be combined by the digital signal 
processing portion of the imager to produce a pair of 
identical lines each displaying detail in both light and 
dark areas of the screen. Instead of alternate high/low 
30 gain, other arrangements could be made. For example every 
third or fourth etc. line could be a low gain or high gain 
line. Similarly, the processing performed .pn the high and 
low gains may be more sophisticated than simple addition. 
This aspect of the invention reduces the vertical 
35 resolution of the image. However, this is not a serious 
problem with low light level images where resolution is 
already limited by a number of other ' factors . 
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As an alternative, the gain could be varied such that 
it is set low for one image (field) and then high for the 
next image (field). As in the line by line example, this 
1:1 low/high ratio could be varied. The two images can 
then be combined in a similar' manner to the line by line 
range although the frame update rate is reduced by half 
for a 1:1 ratio. 

Various modifications to the embodiments described 
above are possible and will occur to those skilled in the 
art. Although described in relation to CCD images using 
multipliers, the vertical (frame transfer) smear 
compensation techniques and dynamic range extensions 
technique are not limited to such arrangements and may be 
implemented on conventional CCDs. Similarly, none of the 
aspects of the invention described are limited to CCD 
arrays in which lines or frames are stored before being* 
output to an output register. 

Embodiments of the invention in its various .aspects 
have a number of advantages. Each of the first and second 
aspects reduce or eliminate the effects of vertical sme^r 
by processing the image after acquisition. The two aspects 
may be used together or separately. Furthermore, the first 
aspect of the invention may also' reduce the effects of 
horizontal smear due to charge retention in the 
multiplication register. This is especially advantageous 
at the low light levels to which CCD imagers using 
multipliers are particularly well suited. The third aspect 
of the invention has the advantage of increasing the 
dynamic range of the CCD array. This is beneficial in its 
own right but is also advantageous in increasing the 
usefulness of the smear compensation techniques by 
reducing the risk of overloading the output from 
individual CCD cells. 

Various other modifications are possible and will 
occur to those skilled in the art without departing from 
the scope of the invention- which ..is defined by the 
attached claims. 
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CLAIMS 

1. Apparatus for compensating image signals produced by 
a CCD imager for smears, comprising: 

an image data analyser for detecting the 
boundaries of a smear; and 

an image data replacer for replacing data 
between the boundaries of a detected smear with 
alternative image data. 

2. Apparatus according to claim 1, wherein the image 
data analyser comprises means for detecting the boundaries 
of a smear by detecting rates of change in the image data 
greater than a predefined limit 

3. Apparatus according to claim 1 or 2, wherein the 
alternative image data is derived from image pixels 
neighbouring the detected smear. 

4. Apparatus according to claim 1 or 2, wherein the 
alternative image data is derived by interpolation of 
image data. 

5. Apparatus according to claim 2, 3, or 4, wherein the 
image data analyser analyses the image line by line to 
detect intensity gradients greater than the predefined 
limit. 

6. Apparatus according to any preceding claim, wherein 
the image analyser comprises a kernel for analysing a 
portion of the image data at a time. 

o. 

7. Apparatus according to claim 6, wherein the kernel 
has a sliding window to define the portion of the image 
being analysed by the kernel and moveable across the image 
to analyse the complete image . 
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8. Apparatus according to any preceding claim, 
comprising a temporal integrator for integrating at least 
two images acquired by the CCD imager prior to boundary 
detection by the image data analyser. 

5 

9. Apparatus according to any preceding claim, 
comprising a coordinate extractor for extracting the 
coordinates of the smear boundaries and providing the 
extracted coordinates to the image data replacer. 

10 

10. Apparatus according to any preceding claim, wherein 
the image analyser detects the boundaries of vertical 
smears produced on transfer of image data from pixels of 
the CCD imager. 

15 

11. Apparatus according to any preceding claim, wherein 
the CCD imager includes a multiplication register for ' 
multiplying charge produced by the CCD imager and the ;V 
image analyser' detects the boundaries of horizontal smears 

20 produced by incomplete charge transfer in the 
multiplication register. 

12. Apparatus according to any preceding claim, wherein 
the CCD imager comprises an image area having a plurality 

25 of rows of pixels for accumulating charge during an image 
acquisition period, each row corresponding to a line of 
the image, an output register for receiving accumulated 
charge row by row during a transfer mode to produce line 
signals, and at least one row of pixels masked from 

30 incident radiation arranged on the side of the image area 
opposite the output register, the apparatus comprising a 
subtractor for subtracting signal corresponding' to charge 
accumulated during the transfer mode and transferred to 
the output register as lines corresponding to the at least 

35 one masked row, from the line signals corresponding to the 
rows of the image area. 
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13. Apparatus for compensating for frame transfer smear 
in a CCD imager, the imager comprising an image area 
having a plurality of rows of pixels for accumulating 
charge during an image acquisition period, each row 
corresponding to a line of the image, an output register 
for receiving accumulated charge row by row during a 
transfer mode to produce line signals, and at least one 
row of pixels masked from incident radiation and arranged 
on the side of the image area opposite the side on which 
pixels are transferred to the output register, the 
apparatus comprising a subtractor for subtracting a signal 
corresponding to charge accumulated during the transfer 
mode and transferred to the output register as lines 
corresponding to the at least one masked row, from the 
line signals corresponding to the rows of the image area. 

14. Apparatus according to claim 12 or 13, wherein the 
CCD imager further comprises a store arranged between the 
image area and the output register. 

15. Apparatus according to claim 12, 13 or 14, wherein 
the CCD sensor comprises a plurality of masked rows. 

16. Apparatus according to claim 15, wherein a line 
signal is generated corresponding to each masked row and 
the error signal is generated from an average of the 
masked row line signals. 

17 . Apparatus according to any preceding claim, wherein 
the CCD imager comprises a multiplication register for 
multiplying charge received from the output register, the 
apparatus^ comprising means for varying the gain of the 
multiplication register for selected images or portions of 
images. 

18. Apparatus for controlling a CCD imager, the imager 
comprising an image area for acquiring charge from 
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incident radiation, an output register for receiving 
acquired charge from the image area during at transfer 
mode, and a multiplication register for multiplying charge 
received from the output register, the apparatus 
5 comprising a gain controller for controlling the gain of 
the multiplication register to vary the gain of the 
multiplication register for selected images or portions of 
images. 

10 19. Apparatus according to claim 17 or 18, wherein the 

gain is adjusted to be relatively high and relatively low 
on alternate lines of the image. 

20. Apparatus according to claim 19, wherein the gain is 
15 adjusted to be relatively high and relatively low on 

alternate images. 

21. A CCD imaging apparatus comprising apparatus >/ 
according to any of claims 1 to 20. 

20 

22. A CCD camera comprising a CCD imager and apparatus 
according to any of claims 1 to 20. 

23. Apparatus for compensating image signals produced by 
25 a CCD imager for smears, substantially as herein described 

with reference to figure 2, figure 3 or figure 4 of the 
accompanying drawings. 

24. A CCD imager comprising apparatus for compensating 
30 image signals produced by a CCD imager for smears, 

substantially as herein described with reference to figure 
2, figure 3 or figure 4 of the accompanying drawings. 

25. A CCD camera comprising a CCD imager and apparatus 
35 for compensating image signals produced by a CCD imager 

for smears, substantially as herein described with 
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reference to figure 2, figure 3 or figure 4 of the 
accompanying drawings. 
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ABSTRACT 

(Figs 16 2) 

SMEAR REDUCTION IN CCD IMAGES 

Vertical and horizontal smear in images produced by a 
CCD imager can be compensated for by detecting the 
boundaries of the smear and substituting data from an 
alternative part of the image, or interpolated data within 
those boundaries. Vertical, or frame transfer, smear can 
also be compensated for by producing an error signal* from 
charge accumulated during frame transfer and corresponding 
to one or more masked rows of the CCD imager. This error 
signal can be subtracted from the line signals 
corresponding to each row of the CCD images receiving 
incident radiation during image acquisitions. The CCD 
imager includes a multiplication register. The dynamic : 
range of the imager may be extended by varying the gain of 
the multiplication register, for example on a line-by-line 
or frame-by-frame basis. 
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